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Introduction 


In  this  work  during  this  grant  period,  the  investigators  explored  issues  involving  the  use  of 
neural  nets  for  object  recognition.  The  PI,  Gene  Gindi  moved  from  Yale  to  Stony  Brook  late  in 
the  grant  period,  and  continued  the  brunt  of  the  work  at  Stony  Brook  under  a  subcontractual 
arrangemnt  with  Yale.  A  colleague,  Eric  Mjolsness,  became  the  titular  PI  during  this  period, 
and  a  postdoc  involved  in  this  work,  Anand  Rangarajan,  stayed  at  Yale  but  had  his  salary 
partially  paid  during  the  SUNY  suncontractual  period.  This  final  progress  report  is  authored 
by  Gene  Gindi.  Tliis  grant  was  a  continuation  of  a  previous  AFOSR  funded  grant  (  F49620-88- 
C-0025)  that  ran  I'rom  12/88  to  3/90.  The  content  of  the  final  progress  report  for  that  grant  is 
not  duplicated  here.  All  publications  refer  to  the  current  grant  only. 

We  summarize  progress  via  publications  that  were  created  during  the  grant  period.  Below  we 
give  a  brief  summary  of  each.  Reprints  are  attached.  Following  the  discussion  of  each  article, 
we  offer  a  brief  discussion  placing  the  work  in  context. 


Publications  Associated  with  Grant 

1.  J  Utans  and  G  Gindi:  “Improving  Convergence  in  Hierarchical  Matching  Networks  for 
Object  Recognition  ”,  Advances  in  Neural  Information  Processing  Systems  V,  pp.  401  - 
408,  C  Giles,  S  Hanson  and  J  Cowan,  eds,  Morgan  Kaufman,  San  Mateo,  1993. 

A  basic  theme  that  we  have  pursued  involves  object  recogition  through  a  neural  net  that 
attempts  to  match  model  components  to  object  components  through  a  match  network 
reminiscent  of  the  Hopfield  network  for  the  TSP  problem.  To  perform  other  than  trivial 
matching,  notions  of  abstraction  and  hierarchy  need  be  incorporated  into  the  network. 
(The  basic  theme  is  described  in  detail  in  publication  2  below.)  One  not  unexpected 
problem  is  computational;  like  other  complex  network  problems,  this  one  has  problems 
converging.  The  object  recognition  problem  is  difficult  in  that  it  involves  coupled  prob¬ 
lems  of  grouping,  segmentation  and  matching.  This  NIPS  publication  describes  methods 
of  initalizing  the  networks  so  <is  to  improve  convergence.  A  given  class  of  problems  reduces 
to  a  weighted  match  with  parameter  -  dependent  weights.  Here  we  show  that  we  can 
first  solve  for  estimates  of  the  parameters  instead  of  solving  for  them  simultaneously.  The 
resulting  networks  converge  more  quickly. 

2.  G  Gindi,  E  Mjolsness,  P  Anandan:  “Neural  Networks  for  Model-Based  Recognition”,  in 
Neural  Networks:  Concepts,  Applications  and  Implementations,  vol  III,  P.  Antognetti  and 
V.  Milutinovic’,  eds.,  pp.  144  -  173,  Prentice  -  Hall,  1991. 

This  is  book  chapter  that  summarizes  work  by  us  and  collaborators  over  the  years  and 
give  a  detailed  exposition  of  the  model  matching  approach.  It  gives  the  basic  approach  of 
reformulating  model  matching  as  an  optimization,  and  shows  how  to  incorporate  notions 
of  hierarchy  via  pointer  matrices  and  other  data  structures.  The  result  is  an  elaborate  ob¬ 
jective  function  that  is  expressive  enough  to  include  features  of  the  sort  of  AI  frame-based 
recognition  system  not  usually  expressed  as  an  optimization.  A  discussion  of  optimization 
techniques  is  given,  where  constraints  may  be  expressed  either  as  soft  (additive)  constraints 
or  hard  constraints.  The  dynamics  of  the  networks  follow  Hopfield  dynamics  or  can  be 
modified  to  include  hard  constraints.  Experiments  in  the  recognition  of  simple  stick  figures. 


and  in  line  grouping  ,  are  performed. 

3.  G  Gindi,  and  J  Utans  :  “A  Neural  Network  Approach  to  Object  Recognition  and  Image 
Partitioning  Within  a  Resolution  Hierarchy”,  Proc.  SPIE  Intelligent  Information  Systems, 
Orlando,  April  1992. 

This  was  a  poster  presented  at  a  conference  that  illustrated  the  problem  of  segmenting 
a  complex  multipart  object  by  first  classifying  it  into  a  broad  catgory  and  using  the  result¬ 
ing  match  to  index  into  specific  criteria  for  segmenting  further  parts. 

4.  G  Gindi,  and  J  Utans  :  “A  Neural  Network  Performs  Context-Guided  Segmentation  of 
Shapes”,  Proc.  Snowbird  Conference  Neural  Networks  Computing,  Salt  Lalce  City  UT, 
1992. 

This  was  a  Snowbird  poster  similar  to  the  poster  above. 

5.  J  Utans:  “Neural  Networks  for  Object  Recognitin  within  Compositional  Hierarchies”, 
Ph.D.  thesis.  Dept  Electrical  Engineering,  Yale  University,  1992. 

Mr.  Utans,  in  his  Ph.D.  thesis  (G.  Gindi  thesis  advisor)  explores  several  issues  in  hierchical 
optimizing  nets  for  object  recognition.  Experiments  in  simple  systems  that  perform  seg¬ 
mentation,  grouping,  and  matching  in  a  coupled  fashion  are  reported.  Much  of  the  work 
of  other  publications  appears  in  somewhat  more  detailed  form  here. 

6.  G  Shumaker:  “  Neural  Networks  for  Object  Recognition  via  Graph  Matching”,  M.D  The¬ 
sis,  Yale  University  School  of  Medicine  ,  1991. 

In  this  masters  level  (actually  MD)  thesis.  Mr.  Shumadcer  performed  detailed  experi¬ 
ments  on  the  recognition  of  articulated  stick  figures.  He  explored  different  ways  to  express 
constraints  (hard  vs.  soft),  showed,  computationally,  the  advantage  of  an  ISA  special¬ 
ization  hierarchy.  Of  interest  here  is  ani  intil  attempt  to  incorporate  learning  into  the 
network.  Our  previous  efforts  were  elaborate  hardwired  optimization  nets.  Here,  he  at¬ 
tempts,  with  some  success,  learning  of  recognition  parameters  through  the  use  of  modified 
backpropagation  network. 

7.  G  Gindi,  A  Rangarajan  and  G  Zubal  :  “Atlas-Guided  Segmentation  of  Brain  Images  via 
Optimizing  Neural  Networks”,  Proc.  SPIE  Biomedical  Image  Processing  IV,  San  Jose, 
February  1993. 

8.  G  Gindi,  A  Rangarajan  and  G  Zubal :  “Neural  Network  for  Model  Based  Segmentation  of 
MR  Brain  Images”,  pp.  90  -  93,  Proc.l2th  Southern  Biomedical  Engineering  Conference, 
New  Orleans,  April,  1993. 

9.  A  Rangarajan  and  G  Gindi:  “Optimizing  Networks  Applied  to  Atleis  -  Guided  Segmen¬ 
tation  of  Brain  Images  ”,  Proc.  Snowbird  Conference  on  Neural  Networks  for  Computing, 
Salt  Lake  City  UT,  1988. 

The  above  two  articles  and  one  Snowbird  poster  apply  the  model  matching  strategy  to  a 
medical  domain  in  which  a  model  consists  of  stored  shapes  reflecting  normal  brain  anatomy, 
ie.  an  atlcis.  This  is  to  be  used  to  guide  the  segmentation  of  a  Magnetic  Resonance 
brain  scan  into  anatomical  regions.  Of  interest  is  that  this  domain  has  a  shape  regularity 
amenable  to  modelling,  yet  exhibits  enough  variation  to  be  interesting  and  not  trivial.  ( 


Also,  successful  segmentation  turns  out  to  be  quite  useful  in  a  variety  of  applications.  ) 
Here,  the  model  consists  of  bounding  contour  information  as  stored  in  an  atlas.  Contours 
on  the  test  image  are  adjusted  until  they  best  match,  according  to  an  objective  function, 
the  model  and  are  also  consistent  with  the  underlying  2-D  greyscale  image.  In  this  setting, 
model  matching  becomes  a  sort  of  model  -  guided  snake.  (Here,  snake  is  used  in  the  sense 
of  deformable  contour  popular  in  the  computer  vision  community.)  Match  neorons  appeax 
still  as  variables  that  match  elements  of  both  contours. 

10.  G  Gindi,  C  Darken,  K  O’Brien,  M  Stetz  and  L  Deckelbaum:  “Neural  Network  and  Con¬ 
ventional  Clcissifiers  for  Fluorescence  Guided  Ltiser  Angioplasty  ”  ,  pp.  453  -  463,  IMIA 
Yearbook  of  Medical  Informatics  1992,  Advances  in  Interdisciplinary  Science,  Inti.  Medical 
Informatics  Society,  J.  van  Bemmel,  A.  T.  McCray,  eds.,  Schatttauer  Pub.  Co.,  Geneva, 
1992.  (this  article  originally  apeared  in  IEEE  Trans  Biomed  Eng  vol  38,  pp.  246  -  252 
1991.  ) 

A  conventional  neural  net  application  for  classification  of  optical  spectra. 


Discussion: 

The  work  conducted  during  the  three-year  duration  of  the  grant  has  resulted  in  the  ex¬ 
position  of  an  approach  to  general  recognition  problems.  The  approach  attempts  to  unify 
the  disparate  problems  of  segmentation,  grouping,  and  matching  into  a  unified  framework 
expressed  as  an  optimization.  Neural  nets  are  good  at  optimization,  so  merely  recast¬ 
ing  the  vision  problem  in  this  sense  allows  for  the  possiblity  of  fast  analog  nets  to  solve 
complex  problems  in  vision.  There  is  no  magic  in  reducing  the  complexity  of  these  prob¬ 
lems;  the  complexity  is  merely  pushed  onto  the  objective  function,  amd  expressed  as  the 
conventional  difficulties  associated  with  optimizing  complicated  objectives.  However,  the 
uniform  notation  allows  for  progress.  Complex  vision  problems  that  are  still  simpler  than 
the  most  general  vision  problems  may  indeed  be  amenable  to  this  sort  of  attack.  In  earlier 
work,  problems  in  industrial  part  recognition  was  attacked.  The  incorporation  of  better 
optimization  techniques  and  learning  may  also  help. 


We  express  our  gratitude  for  the  AFOSR  support. 
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